Psychological Review

The ability to evaluate ones own knowledge states is often studied using paradigms in which participants make a decision and subsequently report their confidence. This structure has motivated hierarchical models in which confidence arises from a metacognitive process, distinct from the decision process itself, that estimates the probability that the choice is correct (Meyniel et al., 2015; Pouget et al., 2016; Fleming and Daw, 2017). Here, we contrast this framework with an alternative based on an intentional architecture (Shadlen et al., 2008). In this account, choice and confidence are determined simultaneously through a multidimensional drift-diffusion process, where each dimension represents one choice-confidence combination (Ratcliff and Starns, 2009, 2013). Choice, response time, and confidence jointly emerge when one of these accumulators reaches a decision bound. To adjudicate between these accounts, we fit both models to behavioral data from two perceptual tasks: a random-dots motion discrimination task with incentivized confidence reports, and a luminance discrimination task without feedback or incentives. The integrated model provided a superior fit for the incentivized motion task, whereas the hierarchical model more accurately captured behavior in the un-incentivized luminance task. These results suggest that confidence does not rely on a single computational mechanism, but rather its implementation may adapt to the specific demands and structure of the task.

Humans have a remarkable ability to act efficiently and accurately in familiar situations while remaining flexible in novel circumstances. Nonparametric contextual inference has been proposed as a computational principle that can model how agents achieve flexible yet stable behaviour in dynamic and possibly unknown environments. However, it remains an open question how humans learn, deploy and reuse stable contextual task representations so efficiently. To address this question, we propose the nonparametric Bayesian Contextual Control (NP-BCC) model, which integrates nonparametric contextual learning with two well-established cognitive mechanisms: repetition-based automatisation and schema-like prior knowledge. These two mechanisms are assumed to support behavioural stability and facilitate novel task acquisition. Simulations in dynamic multi-armed bandit tasks of increasing difficulty illustrate how the NP-BCC can acquire and reuse contextual task representations, with the proposed mechanisms operating in the intended, functionally meaningful manner. Specifically, we show via simulations that automatisation not only enhances task performance but also stabilizes contextual inference and structure learning, while structured prior knowledge accelerates the acquisition of novel contexts. We discuss the implications of our findings for computational accounts of adaptive behaviour and contextual learning, and outline directions for future empirical work, including investigations of context-dependent behavioural dysregulation relevant to conditions such as substance use disorders. Author summaryPeople are very good at repeating well-learned actions in familiar situations, but they can also quickly adjust their behaviour when circumstances change. How the brain balances stability and flexibility is still not fully understood. There is growing evidence that the brain organizes experience into different "contexts", which are mental representations of encountered situations. Computational models based on this idea can in principle reproduce flexible behaviour, but they often become unstable in complex environments. To improve stability, we borrow two simple strategies from everyday human behaviour. First, people tend to repeat actions that have worked well before. Second, when facing something new, they often reuse strategies from similar past situations. Using simulations, we show that combining these strategies with context-based learning produces more reliable behaviour in the model. Prior experience helps the model understand new situations more quickly, while repeated actions help stabilise behaviour once a situation becomes familiar. Taken together, our findings show how such mechanisms can give rise to both flexible and stable behaviour in the model.

Computational models and neurobehavioral data suggest that encoding variability affects forced-choice mnemonic discrimination. Here, we experimentally manipulated encoding variability on the forced-choice Mnemonic Similarity Task by varying stimulus repetitions during encoding. We first generated predictions from a global matching model. Behavioral data supported all predictions. Across most conditions, repetitions consistently enhanced mnemonic discrimination; however, when encoding variability was induced by 3-repetitions of the original version of the non-corresponding lure and 1-repetition of the target during learning, individuals exhibited increased interference. These findings provide further insight into theories of human memory, especially the effect of stimulus repetition on mnemonic discrimination.

Inferring the true cause of noise--distinguishing between volatility (environmental change) and stochasticity (outcome randomness)--is essential for learning in noisy environments. While most studies rely on binary outcomes, previous models are designed for continuous outcome and use ad hoc approximations to handle binary data, introducing theoretical inconsistencies and interpretational issues. Here, we develop a normative framework for inferring the causes of noise from binary feedback that remains faithful to the discrete nature of the generative process and underlying statistical structure. First, we establish a generative model using a state space approach tailored for binary outcomes and derive the corresponding hidden Markov model inference procedure. Second, we introduce a computational model combining the hidden Markov model with particle filtering to simultaneously infer volatility and stochasticity from binary outcomes. Third, we validate predictions through a 2x2 probabilistic reversal learning task with human participants, systematically manipulating both noise parameters. Results show that participants adjust their learning rates consistent with model predictions, increasing learning rates under volatile conditions and decreasing them under high stochasticity. Our theoretical and experimental results offer a principled approach for dissociating volatility and stochasticity from binary outcomes, providing insights into learning processes relevant to typical cognition and psychiatric conditions.

A widely used framework for studying the computational mechanisms of decision making is the Drift Diffusion Model (DDM). To account for the presence of both fast and slow errors in empirical data, the DDM incorporates across-trial variability in parameters such as the drift rate and the starting point. Although these variability parameters enable the model to reproduce both fast and slow errors, they rely on the assumption that over trials each parameter is independently sampled. As a result, the DDM effectively predicts that errors-- whether fast or slow--occur randomly over time. However, in empirical data this assumption is violated, as error responses are often temporally clustered. To address this limitation, we introduce the autocorrelated DDM, in which trial-to-trial fluctuations in drift rate, starting point, and boundary evolve according to first-order autoregressive (AR1) processes. Using simulations, we demonstrate that, unlike the across-trial variability DDM, the autocorrelated DDM naturally accounts for temporal clustering of errors. We further show that model parameters can be reliably recovered using Amortized Bayesian Inference, even with as few as 500 trials. Finally, fits to empirical data indicate that the autocorrelated DDM provides the best account of error clustering, highlighting that computational parameters fluctuate over time, despite typically being estimated as fixed across trials.

Decisions based on evidence accumulated over time require rules governing when to end the accumulation process and commit to a choice. These rules control inherent trade-offs between decision speed and accuracy, which require careful balance to maximize quantities that depend on both like reward rate. We previously showed that, to maximize reward rate, normative decision rules adapt to changing task conditions (Barendregt et al., 2022). Here we used a novel task to examine whether and how people use adaptive rules for individual decisions under a variety of conditions, including changes in decision outcomes across trials and changes in evidence quality both across and within trials. We found that the participants tended to use rules that adjusted, at least partially, to predictable changes in task conditions to improve reward rate, consistent with a rationally bounded implementation of normative principles. These findings help inform our understanding of the extent and limits of flexible decision formation in the brain.

Signal detection theory posits that subjects in two-stimulus, two-choice discrimination tasks decide by comparing random samples of an evidence variable to a static decision criterion. While the core assumptions of the theory have received ample experimental support, it has become evident that the decision criterion is not static but subject to trial-by-trial fluctuations and can be influenced by experimental manipulations. The mechanisms governing the trial-by-trial criterion changes are however not well understood. Here, we report results from five experiments in which we subjected rats to a two-stimulus, two-choice auditory discrimination task. In the first three experiments, we investigated the effects of stimulus presentation ratios and reward ratios and provide clear evidence that the effects of changing reward ratios are more pronounced than those of stimulus presentation ratios. A model-based analysis revealed that this effect was due to more than tenfold higher learning rates when reward ratios were manipulated. In two separate experiments, we investigated the effect of reward density (i.e., global reward rate) on criterion learning but failed to find consistent effects. A systematic comparison of three different trial-by-trial criterion learning models based on detection theory, the matching law, and reinforcement learning showed that no model was able to capture the differential effects of stimulus presentation and reward ratios. We conclude that subjects explicitly represent either prior stimulus probabilities or entire stimulus distributions, and accordingly future models need to represent these factors as well.

The progress of learning is usually quantified by averaging responses across participants and/or multiple trials within a block. However, such approaches obscure the trial-by-trial progress of learning, which has been shown recently to express a rich variety of dynamics. An alternative approach which does not suffer from this problem is the detection and analysis of points of behavioral change, i.e., change-point analysis. Using change-point analysis, we reanalyzed data from human participants in different predictive learning tasks in which learned contingencies underwent reversal. We find that responses of individual participants were more accurately characterized by behavioral change points than the average learning curve. Importantly, change points significantly shifted to later trials during reversal learning indicating that reversal learning is more difficult than the initial learning. In a computational model based on deep reinforcement learning, we show that the change point shift required the replay of previous experiences, which in turn depends on the hippocampus. This finding is consistent with studies showing that lesions of the hippocampus yield faster reversal learning. In summary, we reaffirm the importance of the analysis of single participant responses, show that phenomenological learning rates are slower during reversal learning, and provide a theoretical account for this difference.

Choice overload occurs when an ever-growing number of options impairs decision quality, because evaluating options taxes cognitive resources. We investigated whether reducing cognitive demand could mitigate overload by encouraging greater cognitive effort to achieve optimal choice. We conducted two experiments manipulating cognitive demand in complementary ways: Experiment 1 reduced demand by presenting high-attractiveness sets, and Experiment 2 did so by providing a shortlist tool. In both experiments, participants chose from sets of 6-24 options while their eye-gaze and electroencephalographic (EEG) data were recorded. We found that reducing demand made decisions faster, but did not improve choice performance as set-size increased. Under low-demand conditions, eye-gaze measures revealed narrower search and EEG measures showed reduced working memory engagement per option, together indicating less searching and processing efforts. These results suggest that even with reduced cognitive demand, people coast through easier decisions, conserving effort and leaving the choice overload effect largely intact.

The benefits of routines for daily functioning are widely acknowledged, yet, despite their apparent importance, methods for quantifying routine maintenance and the causes of their disruption remain lacking. Here, we propose a novel means of defining and quantifying routines (transition entropy). Using the transition entropy, we show that routines can be robustly elicited on tasks that require searching through a grid of squares for a hidden target. Over two experiments (N=100 each), we show that use of routines--as quantified by transition entropy--is robustly perturbed by frequent switches between search grids, as locations specific to the currently irrelevant grid become competitive for selection. Using a normative model that tracks task dynamics, we show that disruption to routines can be attributed to reduced sensitivity to the odds of success for completing a task. This suggests that routine maintenance may be disrupted by over-sensitivity to a lack of reward early in routine performance, or increased expectations regarding the utility of pursuing other tasks.

Visual working memory allows for the brief maintenance of information to serve behavioral goals. It has been shown that when the specific action required to serve a future goal is predictable, people can flexibly change a visual memory representation to incorporate an action-based one, demonstrating the goal-oriented nature of visual working memory. Can such flexibility also be observed within the visual domain, between color and space? In this eye-tracking study, participants remembered either a centrally presented color or a spatial position around fixation. Critically, when remembering a color the response wheel was either randomly rotated, or shown at a fixed rotation, on every trial. When fixed, every target color could be associated with a predictable position on the wheel during response. Do people incorporate this added spatial information in their behavior? Participants utilized color-space associations when remembering color: Response initiation happened faster when the color wheel was fixed compared to random, irrespective of whether an action could be planned or not. Next, we showed that gaze was biased towards the position of the spatial memory target during the delay, extending previous work on gaze biases. Importantly, also when remembering a color, gaze was biased towards the anticipated position of that color on the response wheel when it was fixed. Together, our results show a behavioral benefit of added spatial information for color memory, and systematic changes in gaze that reflect flexible utilization of space.

Predictions can alter working memory (WM) representations. However, its effects may have been mischaracterized due to the use of precise predictions in previous experiments, where exact properties of upcoming memory items are cued in advance. Here we investigated a more ecologically valid scenario, in which we assessed the impact of diffuse predictions, where advance cues provided only partial knowledge about the targets. To investigate the resultant nature of the target representations in WM, we performed a series of multivariate analyses of EEG data. Forty participants judged whether a probe grating was rotated clockwise or counterclockwise relative to a memorized orientation, which was either predictable or unpredictable. Each memory item was preceded by a central color cue (red, green, or blue). In half of the trials, two of these (predictive) colors cued two non-overlapping 90{degrees} segments of orientations that the grating was sampled from. Thus, participants knew the range of possible orientations of these items, but not their exact orientation. In the other half of the trials, a third (non-predictive) color was presented, signaling that the item could have any possible orientation. Behavioral results revealed higher accuracy for predictable items, with systematic biases toward the center of the cued segment. EEG results revealed equally successful decoding of orientation for both predictable and unpredictable items during memory encoding. However, cross-condition decoding was significantly weaker than within-condition decoding, suggesting that the encoding format changed between conditions. Representational similarity analysis showed higher similarity between predictable items, with a representational bias towards the cued segment. Covariance matrices showed lower variance for predictable items while the representational space of predictable items was shrunk. These effects were absent during the maintenance phase. Together, our findings suggest that diffuse predictions alter the geometric layout of the neural representations and stabilize the neural code during WM encoding.

Suddenly finding the solution to a problem after a period of impasse often comes with a feeling of insight. This subjective experience is proposed to arise as a consequence of prediction errors. Accordingly, previous studies have revealed that more incorrect initial predictions result in more intense insights. Crucially however, prominent models of Bayesian inference suggest levels of computationally-defined surprise are not a simple feature of distance between predictions and inputs, but also their precision or certainty. Yet, how these two factors interact to give rise to insight experiences remains unknown. In this pre-registered study, participants were exposed to ambiguous images while they tried to guess the correct label of the image (to derive prediction accuracy) and rated their confidence in that label (for prediction uncertainty). We then measured the intensity of their insight when a solution was given. As predicted, we found that the intensity of insight was a result of both the prediction accuracy and the uncertainty awarded to it. More specifically, when initial predictions were far from the true label, those made with lower confidence induced weaker insights, while the opposite pattern was observed when predictions were closer to the reality. Trial-by-trial estimations of prediction errors from participants responses closely mirrored insight ratings. Finally, we analysed data from two additional independent datasets with different modalities and setups and replicated the interaction between prediction accuracy and uncertainty on the intensity of insight. Altogether, these findings suggest that insight experiences are read out from prediction errors and highlight the key role of uncertainty in characterising this relationship.

Human decisions under uncertainty are shaped by experience, but the computations that translate expectation and experience into choice remain debated in neural and cognitive science. Prior studies highlight reinforcement learning (RL) as a unifying framework, yet it is unclear whether human behavior under risk is better captured by symmetric updating from outcomes or by asymmetric learning that weights reward and loss differently. This work examines which learning strategies better explain trial-by-trial choices given contextual uncertainty and manipulations of outcome distributions. Our results show that a Risk Sensitive (RS) model with asymmetric learning rates best explains human behavior in our novel decision-making task. Fitting candidate models to individual trial histories yielded value signals that predicted both choice and response time. These results highlight that RS model, as an asymmetric learning provides a concise and identifiable account of behavior in decision-making under risk tasks.

The ability to build predictive models of the environment fundamentally drives adaptive behavior. Yet, the real-time dynamics of how these internal models are formed and updated remain poorly understood. Conventional methods often rely on indirect, offline measures or noisy motor responses, limiting insight into the fine-grained computational processes underlying learning. Here, we introduce a generalizable, gaze-based analytical framework that directly tracks the trial-by-trial dynamics of expectation formation and updating. Applying this framework to an unsupervised probabilistic learning task, we categorized anticipatory saccades to dissociate prediction errors arising from environmental stochasticity from those reflecting an inaccurate internal model, and quantified how these predictions were iteratively revised. Learners differentiated between these error types: noise-driven errors were more likely to happen, and triggered less updates than errors reflecting insufficient knowledge of the regularity. At the same time, participants exhibited a strong preference to repeat their previous predictions. This repetition bias was amplified when predictions aligned with the underlying regularity, but was also present for non-aligned responses. Critically, updating depended more strongly on whether a prior belief was consistent with the tasks probabilistic structure than on whether the predicted stimulus matched the actual, presented stimulus. These findings suggest that statistical learning may not strongly be driven by errors; rather, it may rely on conservative updating with relatively low learning rate, or, on a Hebbian, repetition-based process. Our framework thus offers a dual contribution: a broadly applicable tool for quantifying real-time expectations, and evidence for a learning strategy that prioritizes model stability in noisy environments.

We propose a computational instantiation of three cognitive stages from the Dot-Linear- Network (DLN) framework, grounded in a compression-efficiency thesis. DLN stages are characterized as graph-structured belief-dependency representations used to evaluate options: Dot as no persistent belief graph (reactive policies with negligible internal state), Linear as a null graph over option beliefs (K independent option estimates with no information sharing), and Network as shared latent structure (a bipartite factor graph in which F latent factors connect to K options), augmented by a temporal exposure state and an explicit structural learning cycle (hypothesis [->] test [->] update/expand). We distinguish two compression targets--option-factor structure (shared components in expected outcomes) and stakes-factor structure (shared drivers of consequence-bearing exposures)-- whose intersection yields jointly efficient actions that simultaneously improve expected outcomes and marginal exposure impact. In a bandit-like simulation (100 seeds, K [isin] { 20, 50, 100, 200 }, F =5), Network policies dominate Linear policies in cost-adjusted utility at large K, with the empirical crossover occurring much earlier than an analytic cost-only prediction (K* = F + cmeta/cparam), revealing that the advantage is primarily statistical (shrinkage-like estimation gains from factor pooling) rather than purely computational. Under stakes, all non-DLN agents--including Linear-Plus agents with identical factor structure and Network-standard agents with hierarchical Bayesian learning--collapse due to unmodeled cumulative exposure, while Network-DLN maintains positive utility. Within-stage consistency tests (two algorithmically distinct agents per stage) confirm that the collapse pattern is determined by representational topology, not algorithmic choice. These results evaluate internal consistency of a DLN-to-computation mapping under explicit assumptions; they do not validate a developmental theory in humans.

Normative social conformity has been proposed to elicit a hedonic reward signal that is dissociable from informational inferences about decision outcomes. If present, such a signal should reinforce not just the decision that preceded it, but also any incidentally co-occurring stimulus features. Alternatively, normative conformity might reflect a non-hedonic imitation algorithm. Across two studies (n=359) we used a non-deceptive multi-participant gambling task in which trial-by-trial information was provided about the selections and monetary payoffs of two other participants facing the same, recurring, options in real time. Consistent with both accounts, and contrary to mere monetary maximization, the probability of staying with a losing option increased with the degree of decision unanimity. However, contrary to the social reward hypothesis, only monetary payoffs modulated the valence of incidental gambling stimuli. A prosocial framing did not significantly alter this pattern of results, which favors an imitative over a hedonic account of normative social conformity.

Multitasking is generally regarded as detrimental to performance. This deterioration effect is typically explained by the interference among tasks due to the limited capacity of information-processing resources, which in turn reduces the performance in each task. Contrary to this general view, we report evidence for a facilitation effect of multitasking on performance. This facilitation effect was observed in multitasking on a handgrip muscular endurance task and cognitive task, which are known to have little interference with each other. Specifically, we found that performance in the endurance task was facilitated with the difficulty of the concurrent cognitive task. This facilitation effect was mediated by additional pupil dilation due to the cognitive task. Increased effort with the difficulty of the cognitive task cannot explain the facilitated performance in the irrelevant endurance task. Instead, they suggest that the cognitive task elevated overall arousal to a level unattainable by the endurance task alone, which in turn facilitated performance in the irrelevant endurance task. To further test this arousal account, we manipulated participants motivation to the cognitive task by reward without changing its difficulty and found the same pattern of results. Thus, it is not effort or motivation specific to the cognitive task but rather overall arousal level that underlies the facilitation effect. These results unveiled a previously overlooked mechanism: a multitasking-induced arousal boost. Our findings suggest that multitasking can facilitate performance when the net effect of adding a concurrent task is governed less by the capacity limitation and more by the elevation of overall arousal.

Habits in humans are commonly studied through outcome devaluation paradigms, but most existing tasks fail to capture the robustness of habitual behavior seen in animal models. I introduce two novel behavioral tasks designed to overcome these limitations. In the first task, ("shooting aliens task", n = 45), I simplified an existing instrumental learning task and implemented a novel intra-block reversal method in which stimulus positions changed unexpectedly within blocks while maintaining the same stimulus-action mappings. Participants also completed a classical devaluation phase with explicit reward changes. In the second task ("hands-attack task", n = 44), which relied on real-life avoidance behavior, devaluation was achieved by reversing reward contingencies and allowing participants to inhibit the dominant avoidance response in favor of a more effortful counterattack. Across both tasks, overtrained conditions led to more errors and longer response times after devaluation, confirming increased insensitivity to outcome change. Intra-block reversals in the shooting aliens task produced stronger habitual signatures than standard whole-block devaluation, revealing a greater cost of overriding automatic responses. In the hands-attack task, even without prior training, participants showed clear markers of habitual behavior, suggesting that real-world action patterns can replicate key features of laboratory habits. Interestingly, participants were more accurate in overriding overtrained responses when attacks were highly familiar, possibly due to enhanced perceptual processing, although this came at the cost of longer response times. These findings introduce two complementary tools that address key limitations in current paradigms: the intra-block reversal increases habit sensitivity without inflating working memory demands, while the hands-attack task captures naturalistic habit expression without artificial training, using a single, ecologically valid session. Both are suited for clinical applications, particularly where time constraints or cognitive load limit the feasibility of traditional approaches.

Animals exposed to pairings of a neutral stimulus with reward acquire a conditioned response to the neutral stimulus. A prominent hypothesis, formalized in the Temporal Difference (TD) learning algorithm, is that animals learn to predict the future reward associated with the neutral stimulus ("value"). Though the TD algorithm does not explicitly specify what drives conditioned responding, a typical assumption is that it reflects the animals estimate of value. In TD learning, value estimates are updated using reward prediction error (RPE, the discrepancy between observed and predicted reward), and are thought to be signaled by the phasic activity of midbrain dopamine neurons. This hypothesis posits that dopamines effects on conditioned responding are mediated entirely by its effects on learning. However, recent experimental and theoretical evidence suggests that dopamine may play a more direct role in modulating conditioned responding. We use a combination of data analysis and computational modeling to probe the relationship between dopamine and conditioned responding. Our results suggest that dopamine directly modulates conditioned responding, in addition to its role in learning. These findings can be captured by a model in which dopamine RPE acts both indirectly (via learning) and directly on conditioned responding.